Large Deviations in the Free-Energy of Mean-Field Spin-Glasses 
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We compute analytically the probability distribution of large deviations in the spin-glass free 
energy for the Sherrington- Kirkpatrick mean field model, i.e. we compute the exponentially small 
probability of finding a system with intensive free energy smaller than the most likely one. This 
result is obtained by computing the average value of the partition function to the power n as a 
function of n. At zero temperature this absolute prediction displays a remarkable quantitative 
agreement with the numerical data. 



In the study of disordered systems nearly all predic- 
tions concern the most likely behavior, but there is also 
considerable interest in developing techniques to compute 
the probability distribution of rare events, i.e. the prob- 
ability of finding systems that have properties different 
from the typical ones. The motivations are various: 

• Systems with behavior different form the most 
likely one may have some special interest. 

• The comparison between analytic predictions in the 
large deviations region and numerical or experi- 
mental data may provide a clear-cut test of the the- 
oretical approach used to compute the most likely 
properties. 

• The properties of large fluctuations may be related 
to other more interesting properties of the system. 

Unfortunately even in the simplest non-trivial case, i.e. 
the Sherrington-Kirkpatrick (SK) infinite range model for 
spin glasses, there is no consensus on the procedure to 
perform such a computation. Everybody agrees that as 
a first step we need to compute the thermodynamic func- 
tion 
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where different systems (or samples) are labeled by J, 
Zj{j3) is the partition function and the bar denotes the 
average over different disordered samples. Indeed it is 
well known that the probability of large deviations is re- 
lated to the function 

The disagreement is in the computation of $(n,/3). 
At n = it can be done using the approach of broken 
replica symmetry (that is known to give the exact re- 
sults) , where it coincides with the most likely free energy 
$(0, /3) = ftyp or equivalently with the average equilib- 
rium free energy f^q = ftyp- 

For n > Kondor 1] in 1983 presented a first com- 
putation of $(n,/3) in the region near Tc using the most 
natural ansatz for replica symmetry breaking (RSB) ob- 
taining <&(«, (3) — ftyp+An^. However it was not possible 
to test directly Kondor prediction because all numerical 
data concern the fluctuations of the ground state energy, 
i.e. the system is at zero temperature. 



Many efforts has been concentrated on the scaling of 
the small deviations of the free energy. Indeed based on 
Kondor's result it was argued in [2] that the small devi- 
ations of the free energy per spin from its mean scale as 
^-5/6 _ Tjjjg prediction has been put to test in a series of 
numerical works 0, Hi, [10, 8, 9] and although all esti- 
mates are smaller than 5/6 nobody has claimed that this 
value is definitively ruled out. However it was difficult 
to test the theory in absence of a quantitative prediction 
(the only prediction being on the exponent, a quantity 
that it is rather difficult to measure in a reliable way) . 

More recently a different replica symmetry breaking 
ansatz was proposed by Aspelmeier and Moore [l^, |Tl[ , 
who found $(n) — ftyp', in their approach the prob- 
ability of large deviations goes to zero faster than 
exp(— AI](/)A^) and the small deviations has to be com- 
puted with an approach that is not related to large devi- 
ations. 

In this letter we concentrate on large deviations. We 
follow Kondor's approach and we extend his computa- 
tion to all temperatures, including T = 0; in this way 
we obtain an absolute prediction for the large deviations 
distribution. We compare our analytic results with the 
numerical simulations done at zero temperature and we 
find a remarkable a gree ment. We also find that the alter- 
native approach [Tol llll| cannot be valid for large positive 
n and there are no compelling reasons for which it should 
be valid at fixed positive n when N goes to infinity. The 
problem of computing the large deviations for the SK 
model at all temperatures is thus solved. 

We start our analysis by defining the sample complex- 
ity AS(/) as the logarithm divided by N of the probabil- 
ity density of finding a sample of size N with free energy 
per spin / in the thermodynamic limit 0, i.e. 
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For large N the majority of the samples has free energy 
per spin equal to ftyp, and all other values have expo- 
nentially small probability. Consistently AI](/) is less or 
equal than zero, the equality holding for / — ftyp, i.e. 
AE(/(yp) = 0. For some values of / it is possible that 
AE(/) = — oo, signalling that the probability of large 
deviations goes to zero faster than exponentially with N. 
It is evident that $(n) is the Legendre transform of 
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AS(/): 

- /3n$(n) = -(3nf + AE(/) , /3n 
Equivalently we have: 

AE(/) = I3nf - , / 
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FIG. 1: The dAT hne in the {T, l3n) plane. The value of /?n 
diverges in the zero-temperature limit as /3n ~ \/— 2 In T, as 
a consequence the function AE(Ae) at zero temperature is 
described by the RSB solution at any value of Ae 



On the dAT line the value of n is ndAT^T) = 4t/3 for 
small T = 1 — T while nj^AriT) vanishes in the zero- 
temperature limit as n = T\J —2 InT. As a consequences 
in the rescaled (T, plane the dAT line never touches 
the T = line and AE(e) at T = is always in the RSB 
phase, see fig ([T]). 

For small n the RS solution is not only unstable but 
also inconsistent, indeed near the critical temperature 
(for T < Tc) we find an unphysical positive value of the 
complexity difference. At any finite temperature $(n) is 
described by the RS solution at large values of n. Both 
above and below the critical temperature, the behavior of 
$(n) for large values of n is $(ri) = — /3n/4 — ln2/(/9n) -|- 
©(e-^/^"). This leads to AE(/) = -/^ + ln2 + o(l) for 
large negative /, note that this is the same behavior of 
the Random- Energy-Model [13|]. 
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Generally speaking for positive n one must distinguish 
two regions in the T — n plane separated by the so called 
de Almeida Thouless (dAT) line, see fig. (H]). In the re- 
gion above the dAT line, the phase is replica-symmetric, 
while replica symmetry is broken below. 

In the Replica-Symmetric (RS) region the order pa- 
rameter is the overlap q. The corresponding value of the 
potential ^{n,q) is given by 

Fn{q) = -^(l-2g+(l-n)g2) + 

- — In / ^e-ft(2cosh/32/)" . 

The overlap q can be computed by solving the equation 
d^{n,q)/dq = 0, that is equivalent to : 

f ^ (cosh Py)" tanh^ f3y dy , , 

q = (5) 

J e ^1 (cosh /??/)" dy 

In the low temperature phase the RS solution is un- 
stable at small values of n. In the (n, T) plane the dAT 
line is specified by the condition 

2 

y2 ^ / (cosh/3y)"(l - tanh' pyf dy 
J e~ 27 (cosh Py)"^ dy 



FIG. 2: Free energy vs. temperature at equilibrium (solid) 
[isj l and on the dAT line (dashed), for small r = Tc - T the 
difference is feg-fdAT = 2t^ /45 + 0{t^). The free energy on 
the dAT line diverges as — -^Z— (lnT)/2 at low temperatures. 

Below the dAT line we must break the replica symme- 
try. Since the free energy at the dAT line is not equal to 
the most likely free energy (ftyp), see fig. we must 
look for a free energy that shows some dependence on n 
also in this region and the one suggested by Kondor is 
the most natural one. 

In Kondor's approach for n < UdAriT) < 1 one in- 
troduces a function q{x) defined for n < a: < 1 that de- 
scribes the breaking of replica symmetry in the low tem- 
perature phase; A functional F„[q] is obtained such that 
$(n) = maxgi^„[g]. The function q{x) that maximizes 
Fn[q] can be found by solving the stationarity equation 
SF/Sq{x) — 0. This generahzes the standard approach 
that is proved to give the correct value of 4>(n) at n = 

The form of the free energy functional is the usual one, 
the only difference being that all functions are defined in 
the interval n < x < 1. One finds that 

Fn[q] = (l - 29(1) + ^ q\x) dx^ + 
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The function f{x,y) is defined in the strip n < a; < 1 
and obeys the following equation: 



/ = -|(/" + WT)- 



(7) 



where dots and primes mean respectively derivatives with 
respect to x and y. The initial condition is on the 
right boundary of the strip, at x = 1, where /3f{l,y) — 
log 2 cosh /Jy. There are many ways in which one can 
compute the maximum of Fn [q] . Here we follow [l6j 
and introduce Lagrange multipliers P{x,y) to enforce 
equation ([7]). The resulting equations are: 



q{x) 
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P{x,y)m^{x,y)dy 
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These are the same equations of the standard n Q 
case, the only difference being in the initial condition for 
P{x, y) that reads: 



P(0, v)^c exp 



{y-hf 
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Pnf{n,y) 
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where c is a normalization constant in order to have 
J P{x,y)dy = 1. Since Fn[q] is extremized with respect 
to q{x), the conjugate variable / can be obtained as the 
partial derivative of nFn[q] with respect to n evaluated 
at the saddle point: 



/ 



l-2q(l)+ / q^{x)dx-7iq^{n) 



-{fin,y)), 

(12) 

where square brackets represent average with respect to 
the measure dfj. = exp(— (y — h)^/2q{n) + (3nf{n,y)). 

Kondor [ij found that near the critical temperature 
^(n) = ftyp - 9n^/5120. We have solved the RSB equa- 
tions and computed q(n, x) and $(n) as a series in pow- 
ers of 72 and T — 1 ~ T [15| up to the 18th order, the 
series is reported in appendix. It can be proved (and it 
is confirmed by the explicit computation) that the lowest 
power of n in the expansion of <i>(n) is and that there 
is no term. For negative n the saddle point of the 
Fn[q] is the standard q(x) corresponding to n = 0, thus 
<I>(n) — ftyp for n < \1t\. The corresponding sample 
complexity as a function of A/ = / — ftyp reads: 

AE(/) = -oo for A/ > 

AE(/) = a6/5|A/l'/' + 0(|A/|«/5) for A/ < 



Where ag/s = — 5/?|c5|~^/^6~®/^ and C5 is the coefficient 
of in the expansion of $(n). 

We have verified by an expansion in powers of n at 
finite temperature that the 0{n^) scaling of $(n) holds 
true at all temperatures as follows from an analytic argu- 
ment that for reasons of space will be reported elsewhere 
[II- 

It is interesting to note that from the third order on, 
all derivatives of <I>(n) (with respect to n,T and both) are 
discontinuous on the dAT line i.e. the transition is third 
order. This is the same behaviorjjf the free energy on 
the dAT line in the {h,T) plane [19} . 

When /3 ^ 00 the complexity AE(/) goes to a well- 
defined limit. Therefore from eq. ^ $(n) is actually a 
function of /3n and the coefficient Ca of n" in the power 
series of $(n) diverges as /S" in the zero temperature 
limit. 

The series in powers of r of C5 (the coefficient in 
$(n)) can be used to obtain its behavior in the whole 
low temperature phase provided one uses the information 
that C5 ~ in the zero-temperature limit. Indeed the 
series can be resummed using Fade approximants with 
estimated errors not greater that 1% in the whole tem- 
perature range. 




FIG. 3: Comparison between the numerical and analytical 
sample complexity at zero temperature, see text. The data 
are those of Ref . [3] . 



From the Fade approximants of C5/3 ^ and Cjf3 ^ we 
estimate cg ~ -0.0060(1) near T = and C7 ~ 
— 0.0150(5)/3^ in the SK model. The zero temperature 
complexity then reads: 

AS = -1.62(1) |Ae|«/^ + 3.1(1) {Aef/"^ + 0(Ae«/^) , 

(13) 

Unfortunately the second term yields a big correction 
to the first one, indeed: i) the exponents of the series 
grow slowly (as (6 -|- z)/5, z = 0, 2, 3, . . . because there is 
no term in <I>(n)) and ii) the coefficients of the series 
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grow quickly with order, actually we expect the series to 
be asymptotic as is usually the case in this context p^ . 

In order to bypass this problem and have a good con- 
trol on AS(Ae) we have adopted a method introduced 
in to obtain q{x,T) from its series in powers of x 
and T. We have transformed the series of AE(A/) in 
powers of A/ and r in a power series of just r by setting 
^/ = (^■s^-|-^r'')c with c a parameter in the range [0, 1]. 
The corresponding series in powers of r were resummed 
for any given c through Pade approximants obtaining the 
curve AI](Ae) in parametric form. By resumming the se- 
ries of $(n, t) as a function of r we have been able to ob- 
tain the sample complexity in the whole low-temperature 
phase using the technique of Pade approximants: 18 or- 
ders of the Taylor expansion give us a very good control 
on the function. 
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FIG. 4: Plot of the numerical complexity (from [^) as a func- 
tion of the energy density for different sample sizes at zero 
temperature. The data have been shifted vertically by an 
amount Aiv so that the complexity vanishes at the typical 
energy Etyp = —.7633. 



Using this technique we find that, for not too large Ae, 
the zero temperature result shown in fig. ([3]) differs by 
less than 1% from the first term of eq. p^ in this range of 
energy differences. We compared the sample complexity 



with the numerical data at zero temperature of Ref. [5| as 
a function Ae and find a very good agreement. For each 
N we have plotted ASat = \n{P{AeN)/N"^^^)/N with 
Acat = e — Cat (the average energy at size N): we put 
the N^^^ factor in the definition so that that AE goes to 
a constant for Ae^ = 0. Unfortunately the dependence 
of e^r on the size is not negligible and the reader may 
wonder what happens if we plot the data at fixed en- 
ergy. This has been done in fig.(|l]), where data (from Q) 
have been shifted vertically by an amount A^r so that the 
complexity vanishes at the typical energy Etyp — —.7633, 
as it should do in the infinite volume limit. This correc- 
tion (Ajv) goes to zero at large TV but it is important for 
having a good scaling for finite N. It is interesting to 
note that for E < Etyp the numerical data approach the 
theoretical prediction from below, thus strongly suggest- 
ing that AS(Ae) is finite at variance with the alternative 
scenario where AE goes to — oo when N goes to infinity. 

Using the standard hierarchical ansatz we have com- 
puted the large deviations function at all temperatures. 
In this way we have been able to confirm that the sam- 
ple complexity AE(A/) is proportional to |A/|^/^ for 
small negative A/, this result strongly suggests that the 
sample-to-sample fiuctuations are proportional to N~^/^. 
We have verified that the numerical data of (Hj are in re- 
markably good agreement with our absolute prediction. 
We believe that our results solves the problem of comput- 
ing the large deviations function for negative A/. One 
could therefore start to study more difficult problems, 
like the large deviation function for positive A/ in the 
SK model. One could also try to extend our results to 
other models, such as Bethe lattices or large dimensions 
short range models, work is in progress in these directions 
0- 
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APPENDIX A: POWER SERIES OF $(n) 

In this appendix we report the power series of ^(n) of 
the SK model in the low temperature phase up to the 
18th order in n and r = 1 — T. At all order in t the 
smallest power of n is and there is no term. 



, 1 T , t2 r3 t5 3r6 79r^ 1679 13679 1728361 r^" 

<I>(n) = In 2 |-rln2 1 1 1 1 

^ ^ 4 4 4 12 24 120 20 140 560 720 12600 

19214684 r" 2741593487 t^^ 3939806687x^3 773933492429 86662083146207 r^^ 



17325 277200 40950 764400 7567560 

139738065304401461 r^^ 45875375549246420713 r^^ 11276190176083149262457 r^^ 



1009008000 25729704000 463134672000 

99 r 27 r2 279 981 351 2799 344241 47010861 r« 



5120 5120 320 1280 2560 400 12800 22400 358400 



36684189 830566899 ri° 1928757352257 r" 98506298782713x^2 8635947355938261 r^^^ 
25600 51200 9856000 39424000 256256000 , 

81 2673 T 7047 35559 75573 1943757 7442847 



V 143360 ~ 143360 ~ 35840 35840 35840 179200 179200 

762113853 r'' 17011569051 210723811119 13663823711841x^0 1027400967213903 r" 

1254400 ^ 2508800 2508800 12544000 68992000 

/ 243 4131 r 15309 34263 429381 2740311 11253573 

V 32768 32768 16384 8192 32768 81920 163840 

107945217 669127959 12126319893 183401224893 r^" 

+ — + 



573440 4587520 2293760 3276800 

60021 1720683 T 3703563 48430143 213993819x4 2813451327 



5734400 5734400 1433600 2867200 5734400 7168000 

2765750427 1836578874951 379740674928681 189083279254923 r'' 



- + 



1146880 50176000 802816000 28672000 

/ 155277 911979 T 36721917 110699379 1837467099 2973858543 



-+ 



V 3276800 819200 3276800 1638400 6553600 3276800 

76627955097 104357662929 853398339489 
32768000 16384000 131072000 ) 

829433601 22603330989 r 7219643481x2 20133254457 1149209550873x4 

-+ 



5046272000 5046272000 180224000 57344000 2523136000 

218994700592277 1073108844538299 _ 61281594304289307 
12615680000 6307840000 22077440000 ) 

/ 131410269 2903445891 r 4533651 22510325169 
V 183500800 183500800 25600 18350080 

1165811276367 5189828163921 89579196304317 t^X 
183500800 229376000 917504000 ) 

15299148393873 14089860473859 r 330886579531671x2 _ 13990469422488399 
~ 5651824640000 209924915200 565182464000 1836843008000 ^ 

20015370592779843 9384066047520578313 



+ - 



1335885824000 10496245760000 
/ 66042560169 580058908857 r 15101741931291x2 
V 5138022400 2569011200 5138022400 

125170590832281 7885818877083003 
^ 6422528000 51380224000 

1557529661529486369 13663672258178594727 r 107529809054090820291 t"^ 



29389488128000000 14694744064000000 14694744064000000 

46100805957050412573 ' 



- + 



262406144000000 



/ 190687314873528513 320621966627776497 r 18912071856450181023 t"^' 
V 723433553920000 180858388480000 361716776960000 ' 

1263734G2(ir)8844234883011 92659942781039()U7442731 



111915170791424000000 55957585395712000000 

10254234479592769713 



1808583884800000 
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